Improve the Efficiency of the /api/index/perms API call#12200
Open
qqmyers wants to merge 2 commits intoIQSS:developfrom
Open
Improve the Efficiency of the /api/index/perms API call#12200qqmyers wants to merge 2 commits intoIQSS:developfrom
qqmyers wants to merge 2 commits intoIQSS:developfrom
Conversation
qqmyers
added
GDCC: QDR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it: The (undocumented?) API call /api/index/perms iterates through all dvobjects in the database in one synchronous transaction, making it `unusable. This PR replaces that logic with an asynchronous iteration over the datasets and dataverses in the root dataverse and use of the index self and children logic used elsewhere. That should make it much faster/less memory intensive.
Which issue(s) this PR closes:
Special notes for your reviewer: FWIW - the old code calls findAll dvobjects which, despite the allExceptFiles variable name, includes files. The code then puts all the files in a map and, as far as I can see, never used them.
Suggestions on how to test this: Run the api call before and after, confirm speed/memory improvement, check the db to make sure all last per index times are updated after the new call.
Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Is there a release notes update needed for this change?:
Additional documentation: